Adaptive Pull-Based Data Freshness Policies for Diverse Update Patterns

An important challenge to e ective data delivery in wide area environments is maintaining the data freshness of objects using solutions that can scale to a large number of clients without incurring signi cant server overhead. Policies for maintaining data freshness are traditionally either push-based or pull-based. Push-based policies involve pushing data updates by servers; they may not scale to a large number of clients. Pull-based policies require clients to contact servers to check for updates; their e ectiveness is limited by the diÆculty of predicting updates. Models to predict updates generally rely on some knowledge of past updates. Their accuracy of prediction may vary and determining the most appropriate model is non-trivial. In this paper, we present an adaptive pull-based solution to this challenge. We rst present several techniques that use update history to estimate the freshness of cached objects, and identify update patterns for which each technique is most e ective. We then introduce adaptive policies that can (automatically) choose a policy for an object based on its observed update patterns. Our proposed policies improve the freshness of cached data and reduce costly contacts with remote servers without incurring the large server overhead of push-based policies, and can scale to a large number of clients. Using trace data from a data-intensive website as well as two email logs, we show that our adaptive policies can adapt to diverse update patterns and provide signi cant improvement compared to a single policy.